Mixing restartable and non-restartable requests with performance enhancements

ABSTRACT

A computer-implemented method includes setting a respective flag in a first buffer of a hardware accelerator. The first buffer includes the respective flag of the first buffer, and a second buffer of the hardware accelerator includes a respective flag of the second buffer. A hardware state of the hardware accelerator is maintained in the first buffer, based on the respective flag of the first buffer being set. A first request directed to the hardware accelerator is received. It is determined that that the first buffer has the respective flag set. The first request is passed to the hardware accelerator, where passing the first request includes passing to the hardware accelerator a pointer to the first buffer, based on the first buffer having the respective flag set.

DOMESTIC PRIORITY

This application is a continuation application of the legally relatedU.S. Ser. No. 16/174,521 filed Oct. 30, 2018, the contents of which areincorporated by reference herein in their entirety.

BACKGROUND

The present invention relates to hardware accelerators and, morespecifically, to mixing restartable and non-restartable requests withperformance enhancements.

Hardware acceleration is the performance of certain functions inhardware, such that those functions may potentially be performed moreefficiently than they might be if performed in software, or so as toenable the software to focus on other functionality. The hardware usedin hardware acceleration may be integrated with a central processingunit (CPU) of a host machine, or that hardware may be a separate deviceknown as a hardware accelerator.

Generally, a hardware accelerator is connected to the CPU, such asthrough an input/output adapter. Software on the host machine uses ahardware accelerator through one or more libraries, each of whichutilize a common interface, through which the software can communicatewith a device driver of the hardware accelerator. In other words, thelibrary communicates with the device driver, which communicates with thehardware accelerator. Requests can be executed on the hardwareaccelerator through the interface. In some cases, however, a requestwill need to be restarted, potentially on a different hardwareaccelerator. This may be the case, for example, if the hardwareaccelerator initially executing the request fails during the execution.

To enable restarts of requests, the interface provides for an inputbuffer and an output buffer, also referred to respectively as an inputstate area and an output state area, which are managed by the devicedriver. Generally, the input buffer and the output buffer are used tomaintain input states and output states of the hardware accelerator. Toenable restartable requests, the input state existing at the beginningof execution of the request needs to be maintained in the input bufferwhile a request is being executed by the hardware accelerator. To thisend, for instance, the device driver copies the input state from theinput buffer to a secondary buffer, and the device driver processes therequest using the secondary buffer. The result of the request is writtento the output buffer. This maintains the input state in pristinecondition in the input buffer. Thus, if the accelerator fails whileprocessing the request, the request can be restarted with its originalinput state.

Because the output buffer contains the current state of the hardwareaccelerator after execution of a request, the library will instruct thedevice driver to swap the input buffer and the output buffer at theconclusion of the request. For instance, a pointer indicating the inputbuffer can be updated to reference the output buffer, and a pointerindicating the output buffer can be updated to reference the inputbuffer. As a result, the output buffer becomes the input buffer for thenext request received. The acts of writing to the current state to theoutput buffer and then swapping the input and output buffers occurs atthe conclusion of each successful request.

SUMMARY

Embodiments of the present invention are directed to acomputer-implemented method for executing requests on a hardwareaccelerator. A non-limiting example of the computer-implemented methodincludes setting a respective flag in a first buffer of a hardwareaccelerator. The first buffer includes the respective flag of the firstbuffer, and a second buffer of the hardware accelerator includes arespective flag of the second buffer. A hardware state of the hardwareaccelerator is maintained in the first buffer, based on the respectiveflag of the first buffer being set. A first request directed to thehardware accelerator is received. It is determined that that the firstbuffer has the respective flag set. The first request is passed to thehardware accelerator, where passing the first request includes passingto the hardware accelerator a pointer to the first buffer, based on thefirst buffer having the respective flag set.

Embodiments of the present invention are directed to a system forexecuting requests on a hardware accelerator. A non-limiting example ofthe system includes a memory having computer-readable instructions andone or more processors for executing the computer-readable instructions.The computer-readable instructions include instructions for setting arespective flag in a first buffer of a hardware accelerator. The firstbuffer includes the respective flag of the first buffer, and a secondbuffer of the hardware accelerator includes a respective flag of thesecond buffer. Further according to the computer-readable instructions,a hardware state of the hardware accelerator is maintained in the firstbuffer, based on the respective flag of the first buffer being set. Afirst request directed to the hardware accelerator is received. It isdetermined that that the first buffer has the respective flag set. Thefirst request is passed to the hardware accelerator, where passing thefirst request includes passing to the hardware accelerator a pointer tothe first buffer, based on the first buffer having the respective flagset.

Embodiments of the invention are directed to a computer-program productfor executing requests on a hardware accelerator, the computer-programproduct including a computer-readable storage medium having programinstructions embodied therewith. The program instructions are executableby a processor to cause the processor to perform a method. Anon-limiting example of the method includes setting a respective flag ina first buffer of a hardware accelerator. The first buffer includes therespective flag of the first buffer, and a second buffer of the hardwareaccelerator includes a respective flag of the second buffer. Furtheraccording to the method performed by the processor, a hardware state ofthe hardware accelerator is maintained in the first buffer, based on therespective flag of the first buffer being set. A first request directedto the hardware accelerator is received. It is determined that that thefirst buffer has the respective flag set. The first request is passed tothe hardware accelerator, where passing the first request includespassing to the hardware accelerator a pointer to the first buffer, basedon the first buffer having the respective flag set.

Additional technical features and benefits are realized through thetechniques of the present invention. Embodiments and aspects of theinvention are described in detail herein and are considered a part ofthe claimed subject matter. For a better understanding, refer to thedetailed description and to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The specifics of the exclusive rights described herein are particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features and advantages ofthe embodiments of the invention are apparent from the followingdetailed description taken in conjunction with the accompanying drawingsin which:

FIG. 1 is a diagram of a request system according to some embodiments ofthe invention;

FIG. 2 is a flow diagram of a method of managing both restartable andnon-restartable requests through the request system, according to someembodiments of the invention; and

FIG. 3 is a block diagram of a computer system for implementing some orall aspects of the request system, according to some embodiments of thisinvention.

The diagrams depicted herein are illustrative. There can be manyvariations to the diagram or the operations described therein withoutdeparting from the spirit of the invention. For instance, the actionscan be performed in a differing order or actions can be added, deletedor modified. Also, the term “coupled” and variations thereof describeshaving a communications path between two elements and does not imply adirect connection between the elements with no interveningelements/connections between them. All of these variations areconsidered a part of the specification.

In the accompanying figures and following detailed description of thedisclosed embodiments, the various elements illustrated in the figuresare provided with two- or three-digit reference numbers. With minorexceptions, the leftmost digit(s) of each reference number correspond tothe figure in which its element is first illustrated.

DETAILED DESCRIPTION

Various embodiments of the invention are described herein with referenceto the related drawings. Alternative embodiments of the invention can bedevised without departing from the scope of this invention. Variousconnections and positional relationships (e.g., over, below, adjacent,etc.) are set forth between elements in the following description and inthe drawings. These connections and/or positional relationships, unlessspecified otherwise, can be direct or indirect, and the presentinvention is not intended to be limiting in this respect. Accordingly, acoupling of entities can refer to either a direct or an indirectcoupling, and a positional relationship between entities can be a director indirect positional relationship. Moreover, the various tasks andprocess steps described herein can be incorporated into a morecomprehensive procedure or process having additional steps orfunctionality not described in detail herein.

The following definitions and abbreviations are to be used for theinterpretation of the claims and the specification. As used herein, theterms “comprises,” “comprising,” “includes,” “including,” “has,”“having,” “contains” or “containing,” or any other variation thereof,are intended to cover a non-exclusive inclusion. For example, acomposition, a mixture, process, method, article, or apparatus thatcomprises a list of elements is not necessarily limited to only thoseelements but can include other elements not expressly listed or inherentto such composition, mixture, process, method, article, or apparatus.

Additionally, the term “exemplary” is used herein to mean “serving as anexample, instance or illustration.” Any embodiment or design describedherein as “exemplary” is not necessarily to be construed as preferred oradvantageous over other embodiments or designs. The terms “at least one”and “one or more” may be understood to include any integer numbergreater than or equal to one, i.e., one, two, three, four, etc. Theterms “a plurality” may be understood to include any integer numbergreater than or equal to two, i.e., two, three, four, five, etc. Theterm “connection” may include both an indirect “connection” and a direct“connection.”

The terms “about,” “substantially,” “approximately,” and variationsthereof, are intended to include the degree of error associated withmeasurement of the particular quantity based upon the equipmentavailable at the time of filing the application. For example, “about”can include a range of ±8% or 5%, or 2% of a given value.

For the sake of brevity, conventional techniques related to making andusing aspects of the invention may or may not be described in detailherein. In particular, various aspects of computing systems and specificcomputer programs to implement the various technical features describedherein are well known. Accordingly, in the interest of brevity, manyconventional implementation details are only mentioned briefly herein orare omitted entirely without providing the well-known system and/orprocess details.

Turning now to an overview of technologies that are more specificallyrelevant to aspects of the invention, as accelerator technologyimproves, there is no longer a requirement to swap the input and outputbuffers from one request to the next. Rather, while a new hardwareaccelerator may not necessarily support restartable requests, it may becapable of executing requests in place. Writing the complete outputstate to the output buffer can be an expensive process, because the fulloutput state may be quite large. Thus, for a newer accelerator only asingle buffer may be required, and the output state need not be writtenin full into an output buffer. Further, a new accelerator is likely tobe more closely connected to the CPU, resulting in reduced latency andincreased throughput. It is desirable to leverage these features.

Turning now to an overview of the aspects of the invention, one or moreembodiments of the invention address the above-described shortcomings ofthe prior art by providing a mechanism to mark, or flag, one of theinput and output buffers to indicate that the buffer includes thehardware state. However, the software state may continue to switch backand forth between the two buffers. In some embodiments of the invention,the device driver is able to recognize, based on the flag, which buffermaintains the hardware state and can therefore pass the hardware stateto the accelerator, regardless of which buffer is currently acting asthe input buffer for maintaining the present software state.

The above-described aspects of the invention address the shortcomings ofthe prior art by enabling hardware accelerators to process data in placewhen they are capable of doing so while maintaining backwardcompatibility with libraries that switch the input and output buffersbetween requests. By keeping the hardware state in a single buffer,latency can be significantly reduced, as the hardware state tends to besignificantly larger than the software state. Further, device drivers ofolder accelerators can continue to behave as usual, by writing thesoftware state to the output buffer and then swapping the input bufferand the output buffer after executing a request.

Turning now to a more detailed description of aspects of the presentinvention, FIG. 1 is a diagram of a request system 100 according to someembodiments of the invention. As shown in FIG. 1, the request system 100may include and be associated with a hardware accelerator 110, which maybe installed on, or otherwise integrated with, a host machine. Althoughnot shown in FIG. 1, to take advantage of various features ofembodiments of the invention, the host machine may include at least twohardware accelerators 110, thereby enabling a request to be restarted ona second hardware accelerator 110 after a first hardware accelerator 110fails. However, for the hardware accelerator 110 to operate as intended,it is not required that additional hardware accelerators 110 be presenton the host machine.

Each hardware accelerator 110 may have an associated a device driver130, with which software can communicate via an interface 140,implemented by one or more software libraries 150. The interface 140enables access to a first buffer 160 and a second buffer 160, each ofwhich is capable of maintaining a state of the hardware accelerator 110.More specifically, in some embodiments of the invention, each buffer 160may include reserved space for a flag 170, a software state 180, and ahardware state 190 of the hardware accelerator 110. Alternatively,however, only one buffer 160 may include a reserved space for thehardware state 190, while the other buffer 160 includes space only forthe flag 170 and the software state 180. In either case, however, bothbuffers 160 may include reserved space for the flag 170 and the softwarestate 180.

Each library 150 used to access the hardware accelerator may be a newlibrary 150 or a legacy library 150. Generally, after executing arequest, a legacy library 150 takes the current output buffer 160 andthen switches the input buffer 160 and the output buffer 160, while anew library 150 need not switch the input buffer 160 and the outputbuffer 160. In some embodiments of the invention, the request system 100supports both of these library types.

Generally, embodiments of the invention leverage the capabilities of ahardware accelerator 110 that is more closely connected to the CPU thanare older hardware accelerators 110. As a result, latency can bereduced, and throughput can be increased. Thus, for such a hardwareaccelerator 110 it is desirable to optimize requests on the hardwareaccelerator 110 in order to take advantage of these features. However,conventionally, by writing the full output state to the current outputbuffer 160 at the conclusion of each request, latency is unnecessarilyincreased for new hardware accelerators 110. In some embodiments of theinvention, a hardware accelerator 110 is able to process data in placeand thus does not need a distinct input buffer 160 and output buffer160. However, legacy libraries 150 exist that will expect these twobuffers 160 and will automatically swap them after a request. Thus,embodiments of the invention are backward compatible with such legacylibraries 150 while also supporting new libraries 150 that do not swapthe input buffer 160 and the output buffer 160 but, rather, use only asingle buffer 160.

To this end, as mentioned above, each buffer 160 may include space for asoftware state 180, a flag 170, and a hardware state 190, but only asingle of these buffers 160 actually maintains the current hardwarestate 190. Generally, the hardware state 190 may be much larger than thesoftware state 180, and the flag 170 may be implemented by a single bit.In this disclosure, the portion of a buffer 160 reserved for thehardware state 190 is referred to as an extended buffer, while theportion of the buffer 160 reserved for the software state 180 isreferred to as the standard buffer 160. In some embodiments of theinvention, the software state 180 and the flag 170 are incorporated intothe buffer 160 as a header, while the hardware state 190 takes up theremainder of the buffer 160. However, it will be understood that variousimplementations are available.

Generally, the software state 180 is a current state of softwareassociated with the hardware accelerator 110, while the hardware state190 is a current state of the hardware of the hardware accelerator 110.Typically, the hardware state 190 is significantly larger than thesoftware state 180. In some embodiments of the invention, it isunnecessary for both the buffers 160 to maintain this large hardwarestate 190. Rather, in some embodiments of the invention, the flag 170indicates which buffer 160 maintains the actual hardware state 190. Insome embodiments of the invention, the buffer 160 that does not includehardware state 190 may instead include blanked data, such as a series ofzeroes.

The library 150 that uses the hardware accelerator 110 may initializethe buffers 160 before executing the first request to the device driver130. For instance, this may be performed at the instruction of aninitialization function implemented by a new library 150. To this end,the device driver 130 may set all bits in each buffer 160 to zero on thefirst call. The device driver 130 may then flag one of such buffers 160as maintaining the hardware state 190. More specifically, the devicedriver 130 may set the flag 170 by changing the flag bit from 0 to 1.This flagged buffer 160 may thus be the only buffer 160 that maintainsthe hardware state 190 of the hardware accelerator 110.

A newer hardware accelerator 110 need not utilize both an input buffer160 and an output buffer and may, instead, be capable of performingcomputations in place. Thus, a new library 150, potentially developedwith knowledge of the capabilities of such a new hardware accelerator,need not swap the input buffer 160 and the output buffer 160 afterperforming a request on the hardware accelerator 110. In someembodiments of the invention, when a new library 150 is utilized to runa request on the hardware accelerator 110, the device driver 130 passesto the hardware accelerator 110 the software state 180 written in thecurrent input buffer 160 and a pointer to the hardware state 190 in theflagged buffer 160, which does not move. While the request is executed,the input buffer 160 is updated, and at the conclusion of the request,the input buffer 160 may still include the current software state 180.After the request is executed, in some embodiments of the invention, thenew library therefore does not swap the input buffer 160 and the outputbuffer 160. Thus, the current input buffer 160 may remain the inputbuffer 160 for the next request. As a result, the hardware accelerator110 may operate with reduced latency, because the large hardware state190 need not be copied by being written to the current output buffer160.

In some embodiments of the invention, when the device driver 130receives control again, the device driver 130 knows which buffer 160 isthe current input buffer 160. Further, the device driver 130 maydetermine which buffer 160 has its flag 170 set and, therefore, maydetermine the location of the hardware state 190. Because legacylibraries 150 still swap the input and output buffers 160, the hardwarestate 190 is not necessarily stored in the current input buffer 160. Thenext time a request is received by way of a library, the device driver130 is able to pass the hardware accelerator 110 the correct softwarestate 180, in the current input buffer 160, and the correct hardwarestate 190, in the flagged buffer 160.

Each legacy library 150 may continue to swap the input buffer 160 andthe output buffer 160 after a request is performed. Further, in someembodiments of the invention, a legacy library 150 need not be concernedwith, or even aware of, the existence of the flag 170 and the hardwarestate 190. Rather, upon completing a request, the device driver 130 maywrite to the current output buffer 160, specifically, to the softwarestate 180 of the current output buffer 160, while leaving the flag 170and the hardware state 190 untouched. The legacy library 150 may thenswitch the input buffer 160 and the output buffer 160.

Thus, when the device driver 130 receives control again, the devicedriver 130 has access to the current input buffer 160 as well as thecurrent output buffer 160. The device driver 130 may check whether theinput buffer 160, as indicated by the library 150 after the switch, hasthe flag set. If the flag is set, then the device driver 130 knows thatthe current input buffer 160 maintains the hardware state 190 inaddition to the software state 180. However, if the flag is not set inthe input buffer 160, then the device driver 130 knows that the currentoutput buffer 160 maintains the hardware state 190. As discussed above,after initialization, the placement of the hardware state does notchange, according to some embodiments of the invention.

FIG. 2 is a flow diagram of an example method 200 of managing bothrestartable and non-restartable requests through the request system 100,according to some embodiments of the invention. According to someembodiments of the invention, this method 200 enables a hardwareaccelerator 110 to receive requests via libraries 150 that expectprocessing to be performed in place, without swapping the input buffer160 and the output buffer 160, as well as via libraries 150 that expectto swap the input buffer 160 and the output buffer 160. As a result,restartable requests are supported through the use of two distinctbuffers 160, and performance is improved through leveraging thecapabilities of a hardware accelerator 110 that does not require theswap to occur.

As shown in FIG. 2, at block 205, upon receiving a first request, thedevice driver 130 initializes the buffers 160 associated with thehardware accelerator 110. To this end, for instance, the device driver130 can check whether the flag 170 is set in either the current inputbuffer 160 or the current output buffer 160. If no flag 170 is set ineither buffer 160, then the device driver 130 may set the various bitsin both buffers 160 to zeroes, thus blanking the buffers 160.Additionally, the device driver 130 may set the flag 170 in the inputbuffer 160 to a value of 1. At the conclusion of the request, after theinput buffer 160 has been updated with an output state, including bothan output software state 180 and an output hardware state 190, a subsetof the input buffer 160 may be copied to the output buffer 160. Morespecifically, for instance, this subset may include the software state180, while excluding the flag 170 and the hardware state 190. Thus,after this first request, the input buffer 160 may have a set flag 170and may include both the software state 180 and the hardware state 190,while the output buffer 160 has its flag 170 unset and includes thesoftware state 180 but blanked data in place of the hardware state 190.Although the input buffer 160 is flagged in the above example, and thusmaintains the hardware state 190, one of skill in the art willunderstand that the output buffer 160 could be flagged alternatively. Insome embodiments of the invention, a new library 150 provides aninitialization method, or the like, to instruct the device driver 130 toperform these initialization tasks upon receipt of its first request.

At block 210, after initialization, another request is issued to thehardware accelerator 110 and received at the device driver 130. If therequest is made through a new library 150, then the method 200 mayproceed to block 215. However, if the request is made through a legacylibrary 150, then the method 200 may proceed block 235. It will beunderstood that the device driver 130 need not detect whether thelibrary 150 being used is a legacy library 150 or a new library 150.Rather, depending on the library type, one of these paths will beutilized, according to some embodiments of the invention.

At block 215, when the request has been issued to the hardwareaccelerator 110 through a new library 150, the device driver 130determines which buffer 160 has its flag 170 set. At block 220, thedevice driver 130 adjusts a pointer to the hardware state 190 to pointto the hardware state 190 in the buffer 160 with the set flag 170. Thismay be the input buffer 160 or the output buffer 160, because thebuffers 160 may switch from time to time. At block 225, the devicedriver 130 passes the request to the hardware accelerator 110 forprocessing. At block 230, while completing the request, the hardwareaccelerator 110 writes the resulting software state 180 back to theinput buffer 160. As such, in some embodiments of the invention, theinput buffer 160 maintains the current software state 180, and theflagged buffer 160 maintains the current hardware state 190. The method200 may then return to block 210, where additional requests arereceived.

At block 235, when a new request has been issued to the hardwareaccelerator 110 through a legacy library 150, the device driver 130determines which buffer 160 has its flag 170 set. At block 240, thedevice driver 130 adjusts a pointer to the hardware state 190 to pointto the hardware state 190 in the buffer 160 with the set flag 170. Atblock 245, the device driver 130 passes the new request to the hardwareaccelerator 110 for processing. At block 250, while completing therequest, at the instruction of the legacy library 150, the device driver130 writes the output state to the current output buffer 160. Morespecifically, in some embodiments of the invention, this output stateincludes the software state 180 and does not include a flag 170 or thehardware state 190, as the legacy library 150 is unconcerned with thehardware state 190. Thus, at this point, the output buffer 160 may havethe current software state 180, and the flagged buffer 160 may maintainthe current hardware state 190. At block 255, at the instruction of thelegacy library 150, the device driver 130 swaps the input buffer 160 andthe output buffer 160. Therefore, in some embodiments of the invention,the new input buffer 160 maintain the current software state 180, andthe flagged buffer 160 maintains the current hardware state 190. Themethod 200 may then return to block 210, where additional requests arereceived.

It will be understood that, regardless of which library 150 is used tomake a request, at the conclusion of processing the request, the inputbuffer 160 maintains the software state 180 and the flagged buffer 160maintains the hardware state 190. Further, it will be understood that,regardless of whether the next request arrives through a legacy library150 or a new library 150, the current input buffer 160 may be used toprovide the software state 180, and the flagged buffer 160 may be usedto provide the hardware state 190. Thus, regardless of which library 150is used, the hardware accelerator 110 will operate as intended. Thisenables a newer hardware accelerator 110 to be used effectively bytaking advantage of reduced latency and thus improved performance, whilealso maintaining backward compatibility, including the support forrestartable requests.

FIG. 3 is a block diagram of a computer system 300 for implementing someor all aspects of the request system 100, according to some embodimentsof this invention. The request systems 100 and methods described hereinmay be implemented in hardware, software (e.g., firmware), or acombination thereof. In some embodiments, the methods described may beimplemented, at least in part, in hardware and may be part of themicroprocessor of a special or general-purpose computer system 300, suchas a personal computer, workstation, minicomputer, or mainframecomputer. For example, and not by way of limitation, the hardwareaccelerator 110 may be integrated with, or installed in, a computersystem 300, which acts as its host machine.

In some embodiments, as shown in FIG. 3, the computer system 300includes a processor 305, memory 310 coupled to a memory controller 315,and one or more input devices 345 and/or output devices 340, such asperipherals, that are communicatively coupled via a local I/O controller335. These devices 340 and 345 may include, for example, a printer, ascanner, a microphone, and the like. Input devices such as aconventional keyboard 350 and mouse 355 may be coupled to the I/Ocontroller 335. The I/O controller 335 may be, for example, one or morebuses or other wired or wireless connections, as are known in the art.The I/O controller 335 may have additional elements, which are omittedfor simplicity, such as controllers, buffers (caches), drivers,repeaters, and receivers, to enable communications.

The I/O devices 340, 345 may further include devices that communicateboth inputs and outputs, for instance disk and tape storage, a networkinterface card (NIC) or modulator/demodulator (for accessing otherfiles, devices, systems, or a network), a radio frequency (RF) or othertransceiver, a telephonic interface, a bridge, a router, and the like.

The processor 305 is a hardware device for executing hardwareinstructions or software, particularly those stored in memory 310. Theprocessor 305 may be a custom made or commercially available processor,a CPU, an auxiliary processor among several processors associated withthe computer system 300, a semiconductor-based microprocessor (in theform of a microchip or chip set), a macroprocessor, or other device forexecuting instructions. The processor 305 includes a cache 370, whichmay include, but is not limited to, an instruction cache to speed upexecutable instruction fetch, a data cache to speed up data fetch andstore, and a translation lookaside buffer (TLB) used to speed upvirtual-to-physical address translation for both executable instructionsand data. The cache 370 may be organized as a hierarchy of more cachelevels (L1, L2, etc.).

The memory 310 may include one or combinations of volatile memoryelements (e.g., random access memory, RAM, such as DRAM, SRAM, SDRAM,etc.) and nonvolatile memory elements (e.g., ROM, erasable programmableread only memory (EPROM), electronically erasable programmable read onlymemory (EEPROM), programmable read only memory (PROM), tape, compactdisc read only memory (CD-ROM), disk, diskette, cartridge, cassette orthe like, etc.). Moreover, the memory 310 may incorporate electronic,magnetic, optical, or other types of storage media. Note that the memory310 may have a distributed architecture, where various components aresituated remote from one another but may be accessed by the processor305.

The instructions in memory 310 may include one or more separateprograms, each of which comprises an ordered listing of executableinstructions for implementing logical functions. In the example of FIG.3, the instructions in the memory 310 include a suitable operatingsystem (OS) 311. The operating system 311 essentially may control theexecution of other computer programs and provides scheduling,input-output control, file and data management, memory management, andcommunication control and related services.

Additional data, including, for example, instructions for the processor305 or other retrievable information, may be stored in storage 320,which may be a storage device such as a hard disk drive or solid-statedrive. The stored instructions in memory 310 or in storage 320 mayinclude those enabling the processor to execute one or more aspects ofthe request systems 100 and methods of this disclosure.

The computer system 300 may further include a display controller 325coupled to a display 330. In some embodiments, the computer system 300may further include a network interface 140 for coupling to a network365. The network 365 may be an IP-based network for communicationbetween the computer system 300 and an external server, client and thelike via a broadband connection. The network 365 transmits and receivesdata between the computer system 300 and external systems. In someembodiments, the network 365 may be a managed IP network administered bya service provider. The network 365 may be implemented in a wirelessfashion, e.g., using wireless protocols and technologies, such as WiFi,WiMax, etc. The network 365 may also be a packet-switched network suchas a local area network, wide area network, metropolitan area network,the Internet, or other similar type of network environment. The network365 may be a fixed wireless network, a wireless local area network(LAN), a wireless wide area network (WAN) a personal area network (PAN),a virtual private network (VPN), intranet or other suitable networksystem and may include equipment for receiving and transmitting signals.

Request systems 100 and methods according to this disclosure may beembodied, in whole or in part, in computer program products or incomputer systems 300, such as that illustrated in FIG. 3.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instruction by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general-purpose computer, special-purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special-purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special-purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdescribed herein.

What is claimed is:
 1. A computer-implemented method comprising: settinga respective flag in a first buffer of a hardware accelerator;maintaining a hardware state of the hardware accelerator in the firstbuffer, based on the respective flag of the first buffer being set;receiving a first request directed to the hardware accelerator;determining that the first buffer has the respective flag set; andpassing the first request to the hardware accelerator, wherein thepassing the first request comprises passing to the hardware acceleratora pointer to the first buffer, based on the first buffer having therespective flag set.
 2. The computer-implemented method of claim 1,wherein: the first buffer is a current input buffer for the firstrequest; a second buffer is a current output buffer for the firstrequest; responsive to executing the first request, the hardwareaccelerator writes a resulting software state to the current outputbuffer; and the computer-implemented method further comprises swappingthe current input buffer and the current output buffer at a conclusionof the first request, wherein the first buffer becomes the currentoutput buffer and the second buffer becomes the current input buffer. 3.The computer-implemented method of claim 2, wherein: the first buffer isa current output buffer for a next request; the second buffer is thecurrent input buffer for the next request; and the computer-implementedmethod further comprises: receiving a second request as the next requestfor the hardware accelerator; determining that the first buffer has therespective flag set; and passing the second request to the hardwareaccelerator, wherein the passing the second request comprises passing tothe hardware accelerator a pointer to the first buffer, based on thefirst buffer having the respective flag set.
 4. The computer-implementedmethod of claim 2, wherein the first buffer is a current output bufferfor a second request, wherein the second buffer is the current inputbuffer for the second request, and wherein the hardware acceleratorwrites a resulting software state to the current input buffer responsiveto executing the second request.
 5. The computer-implemented method ofclaim 1, wherein a software state of the hardware accelerator ismoveable between the first buffer and a second buffer, and wherein thehardware state of the hardware accelerator is fixed in the first buffer.6. The computer-implemented method of claim 1, wherein the hardwareaccelerator supports a restartable request.
 7. A system comprising: amemory having computer-readable instructions; and one or more processorsfor executing the computer-readable instructions, the computer-readableinstructions comprising instructions for: setting a respective flag in afirst buffer of a hardware accelerator; maintaining a hardware state ofthe hardware accelerator in the first buffer, based on the respectiveflag of the first buffer being set; receiving a first request directedto the hardware accelerator; determining that the first buffer has therespective flag set; and passing the first request to the hardwareaccelerator, wherein the passing the first request comprises passing tothe hardware accelerator a pointer to the first buffer, based on thefirst buffer having the respective flag set.
 8. The system of claim 7,wherein: the first buffer is a current input buffer for the firstrequest; a second buffer is a current output buffer for the firstrequest; responsive to executing the first request, the hardwareaccelerator writes a resulting software state to the current outputbuffer; and the computer-readable instructions further compriseinstructions for swapping the current input buffer and the currentoutput buffer at a conclusion of the first request, wherein the firstbuffer becomes the current output buffer and the second buffer becomesthe current input buffer.
 9. The system of claim 8, wherein: the firstbuffer is a current output buffer for a next request; the second bufferis the current input buffer for the next request; and thecomputer-readable instructions further comprise instructions for:receiving a second request as the next request for the hardwareaccelerator; determining that the first buffer has the respective flagset; and passing the second request to the hardware accelerator, whereinthe passing the second request comprises passing to the hardwareaccelerator a pointer to the first buffer, based on the first bufferhaving the respective flag set.
 10. The system of claim 8, wherein thefirst buffer is a current output buffer for a second request, whereinthe second buffer is the current input buffer for the second request,and wherein the hardware accelerator writes a resulting software stateto the current input buffer responsive to executing the second request.11. The system of claim 7, wherein a software state of the hardwareaccelerator is moveable between the first buffer and a second buffer,and wherein the hardware state of the hardware accelerator is fixed inthe first buffer.
 12. The system of claim 7, wherein the hardwareaccelerator supports a restartable request.
 13. A computer-programproduct for executing requests on a hardware accelerator, thecomputer-program product comprising a computer-readable storage mediumhaving program instructions embodied therewith, the program instructionsexecutable by a processor to cause the processor to perform a methodcomprising: setting a respective flag in a first buffer of a hardwareaccelerator; maintaining a hardware state of the hardware accelerator inthe first buffer, based on the respective flag of the first buffer beingset; receiving a first request directed to the hardware accelerator;determining that the first buffer has the respective flag set; andpassing the first request to the hardware accelerator, wherein thepassing the first request comprises passing to the hardware acceleratora pointer to the first buffer, based on the first buffer having therespective flag set.
 14. The computer-program product of claim 13,wherein: the first buffer is a current input buffer for the firstrequest; a second buffer is a current output buffer for the firstrequest; responsive to executing the first request, the hardwareaccelerator writes a resulting software state to the current outputbuffer; and the method further comprises swapping the current inputbuffer and the current output buffer at a conclusion of the firstrequest, wherein the first buffer becomes the current output buffer andthe second buffer becomes the current input buffer.
 15. Thecomputer-program product of claim 14, wherein: the first buffer is acurrent output buffer for a next request; the second buffer is thecurrent input buffer for the next request; and the method furthercomprises: receiving a second request as the next request for thehardware accelerator; determining that the first buffer has therespective flag set; and passing the second request to the hardwareaccelerator, wherein the passing the second request comprises passing tothe hardware accelerator a pointer to the first buffer, based on thefirst buffer having the respective flag set.
 16. The computer-programproduct of claim 14, wherein the first buffer is a current output bufferfor a second request, wherein the second buffer is the current inputbuffer for the second request, and wherein the hardware acceleratorwrites a resulting software state to the current input buffer responsiveto executing the second request.
 17. The computer-program product ofclaim 13, wherein a software state of the hardware accelerator ismoveable between the first buffer and a second buffer, and wherein thehardware state of the hardware accelerator is fixed in the first buffer.